Considerations regarding the Choice of Ranking Multiple Criteria Decision Making Methods

Various methods, known as Multiple Criteria Decision Making Methods (MCDM), have been proposed to assist decision makers in the process of ranking alternatives. Given the variability of available methods, choosing an MCDM ranking method is a difficult task. There are key factors in the process of choosing an MCDM method such as: (i) available time; (ii) effort required by a given approach; (iii) importance of accuracy; (iv) transparency necessity; (v) conflict minimization necessity; and (vi) facilitator’s skill with the method. However, the problem is further increased by the knowledge that the solution of MCDM ranking methods may be sensitive to slight variations in entrance data and, in some cases, might replace the best alternative for the worst when the weightings for the criteria are changed. Some researchers have addressed this problem through different approaches, including the evaluation of MCDM ranking methods in the sense of predicting the initial rankings given by the decision maker. The objective of this study is to propose an empirical experiment to evaluate the propensity for initial ranking prediction of the principal MCDM ranking methods, namely: SAW, TOPSIS, ELECTRE III, PROMETHEE II and TODIM. The study also aimed to assess possible common ranking problems in MCDM methods, such as reversibility, found in the literature. It was found that just up to 20% of initial ranking order was predicted entirely correct by some of the methods. It was also found that just a few methods did not present internal ranking inconsistency. The results of this study and those found in the literature give a warning regarding the choice of MCDM ranking methods. It is suggested that special care must be taken in the choice of methods and, besides axiomatic comparisons, ranking comparisons could be a useful way to enhance the decision making process, since MCDM methods are tools for learning about the problem and do not prescribe solutions.